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Abstract. A tree decomposition of the coordinates of a code is a mapping from the coordinate 
set to the set of vertices of a tree. A tree decomposition can be extended to a tree realization, i.e., 
a cycle-free realization of the code on the underlying tree, by specifying a state space at each edge 
of the tree, and a local constraint code at each vertex of the tree. The constraint complexity of a 
tree realization is the maximum dimension of any of its local constraint codes. A measure of the 
complexity of maximum-likelihood decoding for a code is its treewidth, which is the least constraint 
complexity of any of its tree realizations. 

It is known that among all tree realizations of a code that extends a given tree decomposition, 
there exists a unique minimal realization that minimizes the state space dimension at each vertex 
of the underlying tree. In this paper, we give two new constructions of these minimal realizations. 
As a by-product of the first construction, a generalization of the state-merging procedure for trellis 
realizations, we obtain the fact that the minimal tree realization also minimizes the local constraint 
code dimension at each vertex of the underlying tree. The second construction relies on certain code 
decomposition techniques that we develop. We further observe that the treewidth of a code is related 
to a measure of graph complexity, also called treewidth. We exploit this connection to resolve a 
conjecture of Forney's regarding the gap between the minimum trellis constraint complexity and the 
treewidth of a code. We present a family of codes for which this gap can be arbitrarily large. 



1. Introduction 

Graphical models of codes and the decoding algorithms associated with them are now a major 
focus area of research in coding theory. Turbo codes, low-density parity-check (LDPC) codes, and 
expander codes are all examples of codes defined, in one way or another, on underlying graphs. 
A unified treatment of graphical models and the associated decoding algorithms began with the 
work of Wiberg, Loeliger and Koetter 1291 . QUI , and has since been abstracted and refined under 
the framework of the generalized distributive law fV\, factor graphs fT9l, and normal realizations 
Q,!!]. The particular case of graphical models in which the underlying graphs are cycle-free has a 
long and rich history of its own, starting with the study of trellis representations of codes; see e.g., 
IJS'l and the references therein. 

Briefly, a graphical model consists of a graph, an assignment of symbol variables to the vertices 
of the graph, an assignment of state variables to the edges of the graph, and a specification of 
local constraint codes at each vertex of the graph. The full behavior of the model is the set of all 
configurations of symbol and state variables that satisfy all the local constraints. Such a model is 
called a realization of a code C if the restriction of the full behavior to the set of symbol variables 
is precisely C. The realization is said to be cycle-free if the underlying graph in the model has no 
cycles. A trellis representation of a code can be viewed as a cycle-free realization in which the 
underlying graph is a simple path. 

A linear code C has a realization on a graph Q that is not connected if and only if C can be ex- 
pressed as the direct sum of the codes that are individually realized on the connected components 
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of ^ 0. Thus, there is no loss of generaUty in just focusing, as we do, on the case of reahza- 
tions on connected graphs. In this paper, we will be concerned with tree realizations — cycle-free 
realizations in which the underlying cycle-free graph is connected, i.e., is a tree. 

It is by now well known that the sum-product algorithm on any tree realization provides an exact 
implementation of maximum-likelihood (ML) decoding lfn.ir7l. |[T9l . l|29l . A good initial estimate 
of the computational complexity of such an implementation is given by the constraint complexity 
of the realization, which is the maximum dimension of any of the local constraint codes in the 
realization. Now, distinct tree realizations of the same code have, in general, distinct constraint 
complexities. The treewidth of a code is defined to be the least constraint complexity of any of its 
tree realizations. Thus, treewidth may be taken to be a measure of the ML decoding complexity of 
a code. 

Since trellis realizations are instances of tree realizations, the treewidth of a code can be no larger 
than the minimum constraint complexitjQof any of its trellis realizations. In the abstract of his paper 
lIB l, Forney claimed that "the constraint complexity of a general cycle-free graph realization can be 
[strictly] less than that of any conventional trellis realization, but not by very much." While he 
substantiated the first part of his claim by means of an example, he left the "not by very much" 
part as a conjecture fE^. p. 1606, Conjecture 2]. But he also admitted that none of the arguments 
he gave in support of his conjecture "is very persuasive," and that it is equally plausible that [8] 
Conjecture 3] there exists no upper bound on the gap between the treewidth of a code and the 
minimum constraint complexity of any of its trellis realizations. 

One of the main contributions of this paper is an example that affirms the validity of Forney's 
Conjecture 3. We present, in Section [6l a family of codes for which the difference between the 
minimum trellis constraint complexity and the treewidth grows logarithmically with codelength. 
We conjecture that this is in fact the maximal rate of growth of this difference. Our construction of 
this example is based upon results from the graph theory and matroid theory literatures that connect 
the notions of treewidth and trellis complexity of a code to certain complexity measures defined for 
graphs. 

This paper makes two other contributions, both relating to minimal tree realizations. A mapping 
of the set of coordinates of a code C to the vertices of a tree is called a tree decomposition. A tree 
decomposition may be viewed as an assignment of symbol variables to the vertices of the tree. It is 
known that given a code C, among all tree realizations of C that extend a given tree decomposition, 
there is one that minimizes the state space dimension at each vertex of the underlying tree [7 |. This 
minimal tree realization, an explicit construction of which was also given in f7 |, is unique up to 
isomorphism. 

We give two new constructions of minimal tree realizations. The first construction involves a 
generalization of the idea of state merging that can be used to construct minimal trellis realizations 
ll28l Section 4]. We show that any tree realization of a code can be converted to a minimal realization 
by a sequence of state merging transformations. The state space and constraint code dimensions do 
not increase at any step of this process. From this, we obtain the fact that a minimal realization also 
minimizes the constraint code dimension at each vertex of the underlying tree. 

Our second construction of minimal tree realizations uses extensions of the code decomposition 
techniques that were presented in |17|. The main advantage of this construction is its recursive 
nature, which makes it suitable for mechanical implementation. Also, it is relatively straightforward 
to estimate the computational complexity of this construction. We show that the complexity is 
polynomial in the length and dimension of the code, as well as in the size of the underlying tree, but 
is exponential in the state-complexity of the minimal realization, which is the maximum dimension 
of any state space in the reahzation. 



In the context of trellis realizations, constraint complexity is usually referred to as "branch complexity" or "edge com- 
plexity". We make it a point to avoid this usage, so as not to cause confusion when we define the "branchwidth" of a 
code later in our paper. 
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The paper is organized as follows. In Section |2l we provide the necessary background on tree 
realizations of linear codes. The construction of minimal realizations by means of state merging is 
presented in Section[3] Code decomposition techniques are developed in Section|4l and used in Sec- 
tion |5] to derive a recursive construction of minimal tree realizations. Proofs of some of the results 
from Sections I2H5] are deferred to appendices to preserve the flow of the exposition. Treewidth and 
related complexity measures are defined in Section [6l which also establishes connections between 
these code complexity measures and certain complexity measures defined for graphs. These con- 
nections are used to derive the example of a code family for which the gap between minimum trellis 
constraint complexity and treewidth is arbitrarily large. We also touch upon the subject of codes 
of bounded complexity, observing that many hard coding-theoretic problems become polynomial- 
time solvable when restricted to code families whose treewidth is bounded. Section |7] contains a 
few concluding remarks. 

2. Background on Tree Realizations 

Our treatment of the topic of tree realizations in this section is based on the exposition of Forney 
(3,1111; see also [9|. 

We start by establishing some basic notation. We take F to be an arbitrary finite field. Given a 
finite index set /, we have the vector space = {x = {xi G F, i € /)}. For x G F^ and J Q I, 
the notation x| j will denote the projection (xj, i € J). Also, for J C /, we will find it convenient 
to reserve the use of J to denote the set difference I — J = {i G I : i ^ J}. 

2.1. Codes. A linear code over F, defined on the index set /, is a subspace C C F^. We will only 
consider linear codes in this paper, so the terms "code" and "linear code" will be used interchange- 
ably. The dimension, over F, of C will be denoted by dim(C). An [n, k] code is a code of length 
n and dimension k. If, additionally, the code has minimum distance d, then the code is an [n, k, d] 
code. The dual code of C is denoted by C"*-, and is defined on the same index set as C. 

Let J be a subset of the index set /. The projection of C onto J is the code C\j = {c\j : 
c G C}, which is a subspace of F"^. We will use Cj to denote the cross-section of C consisting of 
all projections c|j of codewords c G C that satisfy c| j = 0. To be precise, Cj = {c\j : c G 
C, c| J = 0}. Note that Cj C C\j. Also, since Cj is isomorphic to the kernel of the projection map 
TT : C — > C|j defined by 7r(c) = c|j, we have that dim(Cj) = dim(C) — dim(C|j). Furthermore, 
projections and cross-sections are dual notions, in the sense that (C| j)-*- = {C^)j, and similarly, 
{Cj)^ = {C^)\j. 

If Ci and C2 are codes over F defined on mutually disjoint index sets Ii and I2, respectively, then 
their direct sum is the code C = C1QC2 defined on the index set hU I2, such that C/^ — ^Ih = 
and C/2 = C\i2 = ^2- This definition naturally extends to multiple codes (or subspaces) Ca, where 
a is a code identifier that takes values in some set A. Again, it must be assumed that the codes Ca 
are defined on mutually disjoint index sets I^, a £ A. The direct sum in this situation is denoted 

by ®aeACa. 

2.2. Trees. A tree is a connected graph without cycles. Given a tree T, we will denote its vertex 
and edge sets by V{T) and E{T), respectively, or simply by V and E if there is no ambiguity. 
Vertices of degree one are called leaves, and all other vertices are called internal nodes. Given a 
V ^V, the set of edges incident with v will be denoted by E{v). 

Removal of an arbitrary edge e from T produces a disconnected graph T — e, which is the disjoint 
union of two subtrees, which we will denote by Tf, and Tg, of T. Note that V{Te) and V{Tf,) form 
a partition ofV{T). 

2.3. Tree Realizations. Let C be a code over F, defined on the index set /. To each i G /, we 
associate a symbol variable Xi, which is allowed to take values in F. 
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Figure 1. A depiction of a tree decomposition {T,lo). Vertices of the tree T aie 
represented by squares. Edges are incident with two vertices, while half-edges are 
incident with only one vertex. The vertex vq has no half-edges incident with it, 
indicating that uj^^{vo) = 0, while the vertex vi has two half-edges incident with 
it, which means that \ijj^^{vi)\ = 2. 



A tree decomposition of / is a pair (T, u>), where T is a tree (i.e., a connected, cycle-free graph) 
and (J : / — > y is a mapping from / to the vertex set of T. Pictorially, a tree decomposition (T, a;) 
is depicted as a tree with an additional feature: at each vertex v such that u>~^{v) is non-empty, we 
attach special "half-edges", one for each index in uj^^{v); see Figure[I] 

At this point, we introduce some notation that we will consistently use in the rest of the paper. 
Given a tree decomposition {T,u>) of an index set /, and an edge e E we define J(e) = 
LU^^{V{Te)) and J(e) = ll>~^ {V {T e)) ■ Thus, J(e) and J(e) are the subsets of / that get mapped 
by uj to vertices in Tg and Te, respectively. Clearly, J(e) and J(e) form a partition of I. 

Recall that E{v), v £ V, denotes the set of edges incident with v in T. Consider a tuple of the 
form (T, (Se, e e E), (C^,, v G V)), where 

• (T, Lu) is a tree decomposition of /; 

• for each e £ E, Se is a. vector space over F called a state space; 

• for each v £ V, is a subspace of F"^ ® {®e(^E{v) ' called a local constraint 
code, or simply, a local constraint. 

Such a tuple will be called a tree model. The elements of any state space Se are called states. The 
index sets of the state spaces Se, e G i?, are taken to be mutually disjoint, and are also taken to 
be disjoint from the index set / corresponding to the symbol variables. Finally, to each e £ E, we 
associate a state variable Se that takes values in the corresponding state space Se- 

A global configuration of a tree model as above is an assignment of values to each of the symbol 
and state variables. In other words, it is a vector of the form {{xi G F, i G /), (sg G 5e, e £ E)). 
A global configuration is said to be valid if it satisfies all the local constraints. Thus, G F, i G 
/), (se G Se, e G E)) is a valid global configuration if for each v £ V, ((xj, i G uj~^{v)), (sg, e G 
E{v))) G Cy. The set of all valid global configurations of a tree model is called the fiill behavior 
of the model. 

Note that the full behavior is a subspace 03 C F^ © (©egs'^e)- As usual, 53 |j denotes the 
projection of *B onto the index set /. If !B| ^ = C, then the model (T, u), {Se, e G E), (C^,, v G V)) 
is called a (linear) tree realization of C. A tree realization (T, lo, {Se, e £ E), (C„, v G V)) of C is 
said to extend (or be an extension of) the tree decomposition (T, uj) of the index set of C. Any tree 
decomposition of the index set of a code can always be extended to a tree realization of the code, 
as explained in the following example. 
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J(e) J(e') 

Figure 2. A trivial extension of a tree decomposition (T, lo) of the index set of a 
code C. At tlie vertex v, we have = C. The state variables at the edges e € -E'(t') 
are copies of the symbol variables indexed by J(e). Dashed ovals represent sub- 
trees. 

Example 2.1. Let C be a code defined on index set I, and let {T,uj) be a tree decomposition of 
I. Pick an arbitrary v V, and define Cy = C. Now, consider the set, E{v), of edges incident 
with V. Removal of any e G E{v) produces the two subtrees Tg and Tg. We specify Tg to be the 
subtree that does not contain the vertex v, and as usual, J(e) = uj~^{y (Te))- For each e € E{v), 
the state space Sg is taken to be a copy of¥'^^^\ The remaining state spaces and local constraints 
are chosen so that, for each e £ E{v), the symbol variables indexed by J(e) simply get relayed 
(unchanged) to the state variable Se,' see Figure\2\ It should be clear that the resulting tree model 
is a tree realization of the code C. This will be called a trivial extension of (T, uj). We will present 
constructions of non-trivial extensions of tree decompositions a little later 

□ □ □ □ □ 

Figure 3. A simple path on five vertices. 

Example 2.2. A simple path is a tree with exactly two leaves (the end-points of the path), in which 
all internal nodes have degree two; see Figure \3\ Let C be a code defined on index set I, and 
let (T,uj) be a tree decomposition of I, in which T is a simple path, and uj is a surjective map 
LO : I ^ V{T). Any tree realization of C that extends {T,uj) is called a trellis realization ofC. 
When LO is a bijection, then any trellis realization extending (T, lo) is called a conventional trellis 
realization. When lo is not a bijection (but still a surjection), a trellis realization that extends (T, lo) 
is called a sectionalized trellis realization. In trellis terminology, the local constraint codes in a 
trellis realization are called branch spaces. The theory of trellis realizations is well established; we 
refer the reader to II28II for an excellent survey of this theory. 

Let *B be the full behavior of a tree model (r,a;, (5e, e G E), (C^, v G V)). We will find it 
useful to define certain projections of 53, other than *B| j for J Q I. Let b = ((xj, i £ I), (sg, e G 
ii^)) be a global configuration in 53. At any given v £ V, the local configuration of b at u is defined 
as 

bl^ = {{xi, i G LO-\v)), (se, e G E{v))). 

The set of all local configurations of *B at u is then defined as 53|^ = {b|^ : b G 53}. By definition, 
53 1 ^ C Cv. Similarly, for F Q E, and b as above, we define the projections b|p = (sg, e G F) 
and = {h\p : b G 53}. Clearly, 53 is a subspace of 0ggp Se. If consists of a single edge 
e, then we simply denote the corresponding projections by b|g and *B|g. The following elementary 
property of the projections will be useful later; a proof for it is given in Appendix lAl 
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Lemma 2.1. Let 53 be the full behavior of some tree realization of a code C, defined on the index 
set I, that extends the tree decomposition (T, uj). Suppose that b G 55 and e (z E are such that 
b|g = 0. Then, h\j £ Cj^^^) C-j^^y 

A tree model (or realization) (T, (5e, e G -E), {Cy, v G V)), with full behavior *B, is said to 
be essential if 5S|g = Se for all e £ E. This definition actually implies something more. 

Lemma 2.2. If the tree model {T,uj, {Sg, e G E), {C^, v G V)), with full behavior is essential, 
then *B|^ = for all v £ V. 

A proof of the lemma can be found in Appendix lAl 

An arbitrary tree model can always be "essentialized". To see this, let T = {T,io, {Se, e G 
E), {Cy, V G V)) be a tree model with full behavior !B. Recall that !B|g is a subspace of Se, 
and *B|j, is a subspace of C^. Define the essentialization of T to be the tree model ess(r) = 
{T,uj, (*B|g, e G E), v G V)). It is readily verified that ess(r) has the same full behavior 

asT. 

2.4. Minimal Tree Realizations. Given a code C and a tree decomposition {T,lu) of its index 
set /, there exists an essential tree realization, {T,uj, (5*, e £ E), {C*, v G V)), ofC with the 
following property [i7il.li8il: 

if (T, (J, {Se, e G E), {Cy, V G V)) is a tree realization of C that extends {T,uj), 

then for all e G -E, dim(5|) < dim(5e). 
This minimal tree realization, which we henceforth denote by (C; T, w), is unique up to isomor- 
phism. More precisely, if {T,lo,{SI*, e G E),{C**, v G F)) is also a tree realization of C with the 
above property (except that S* is replaced by S**), then S* = S** for each e £ E, and C* = C** 
for each v £ V. We will not distinguish between isomorphic tree realizations. 

We outline a construction, due to Forney IH, of M.{C; T, to). For any edge e £ E, the sets J(e) 
and J(e) form a partition of the index set I. Set 

5: = c/(Cj(e)e%)), (1) 

and let 

s: : C^C/{Cj^e)(BCj^e)) (2) 
be the canonical projection map. In other words, for c G C, s*(c) is the coset c + (Cj(e) © ^J(e))- 

Now, let OS be the vector space consisting of all global configurations (c, s*(c)) corresponding to 
codewords c G C, where s*(c) = (s*(c), e G E). It is worth noting that *8|j = C, and furthermore, 
53 = C, since c = implies that s*(c) = 0. 

We can now define for each v £ V, the local constraint 

C: = = {(cL-i(,) , (s:(c), e G E{v))) : c G C} . (3) 

The minimal realization M{C;T,uj) is the tuple {T,u;, {S*, e G E), {C*, v G V)). It may be 
verified that OS is the full behavior of M.{C;T,lo), so that A4{C;T,io) is indeed an essential tree 
realization of C. 

From the definition of S* in ([T]), it is clear that for each e £ E, 

dim(<S*) = dim(C) - dim(Cj(e)) - dim(Cj(g)). (4) 

It is useful to point out that dim(5*) may also be expressed as 

dim(5;) = dim(C|^(,)) + dim(C|j(^)) - dim(C), (5) 

a consequence of the fact that for any J C I, dim(Cj) = dim(C) — dim(C|j). Thus, by the 
uniqueness of minimal tree reaUzations, if T** = {T,uj, {S** , e G E), {C**, v G V)) is a tree 
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J(e') 

Figure 4. Figure depicting the perspective about a vertex f in a tree decomposi- 
tion. Dashed ovals represent subtrees. 

realization of C with the property that for all e £ E, dim(5**) equals one of the expressions in Q 
or ©, then T** is in fact M{C; T, lu). 

Forney 18 ] also derived an expression for the dimension of the local constraints C*. Consider 
any v £ V. For each e G E{v), we specify Te to be the component of T — e that does not contain 
V. As usual, J(e) = uj--'^{V{Te)). Then [8. Theorem 1], 

dim(C:) = dim(C) - Yl dim(Cj(e)). (6) 

eeE{v) 

Forney gave the following bound for dim(C*) 18] Theorem 5]: for any e G E{v), dim(5*) < 
dim(C*) < n(C*) - dim(5*), where n{C*) denotes the length of the code C*. The upper bound 
can be improved slightly. 

Lemma 2.3. In the minimal tree realization M.{C] T, u), we have, for v and e G E(v), 
dim(5:) < dim(C:) < dim(CL-i(,)) + J] dim(5:,). 

e'eE(v)-{e} 



Proof. The upper bound may be proved as follows. Since dim(Cj) = dim(C) — dim(C| j) for any 
J C /, we may write Q as 

dim(C:)= ^ dim{C\j^,))-mv)-l)dimiC)- 

Now, let e G E(v) be fixed. We have 

dim(C:) = dim(C|:;(,)) + {^^i^bie')) - dim(C)) . 

e'£Eiv)-{e} 

However, as can be seen from Figure IH J(e) is the disjoint union of (jJ^^{v) and the sets J(e'), 
e' G E{v) — {e}. Therefore, 

dim(C|j(e)) < dim(C|^_i(^)) + ^ dim(C| 

e'eE{v)-{e} 

and hence, 

dim(C:) < dim(CL_i(,)) + J2 (dim(C|^(,,)) + dim{C\j^^,^) - dim(C)) . 

e'eEiv)-{e} 

The lemma now follows from □ 
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As already mentioned, among all tree realizations of C extending (T, to), the minimal realization 
A4{C;T,Lu) minimizes state space dimension at each edge of the tree T. It is natural to ask whether 
A4{C;T,uj) also minimizes local constraint code dimension at each vertex of T. We will show in 
the next section that A4{C;T,u>) does in fact have the following property: 

if {T,LU, {Se, e € E), {Cv, v G V)) is a tree realization of C that extends {T,uj), 
then for all v£V, dim(Q) < dim(C7^). 

We will deduce this fact from an alternative construction of A4{C;T,lo) that we present next. 

3. A Construction of M{C; T, uj) via State Merging 

The construction we describe in this section takes an arbitrary tree realization F that extends the 
tree decomposition {T,uj) — for example, the trivial extension given in Example 12.11 — and via a 
sequence of transformations, converts F to M-{C;T,lo). These transformations constitute a natural 
generalization of the state-merging process in the context of minimal trellis realizations; see, for 
example, ll28l Section 4]. It would be useful to keep this special case in mind while going through 
the details of the description that follows. 

Let r = (T, a;, {Se, e G E), (C„, v G F)) be an essentia^ tree realization of a code C with 
index set /, and let !B be the full behavior of T. As F is essential, we have that *B|g = 5e for all 
e € E (by definition), and !B|^ = for allv eV (by Lemma [I2l). 

Pick an arbitrary edge e ^ E, and for ease of notation, set J = J(e) and J = J(e). Let W be 
the subspace of Se defined by 

W = {seSe-. 3bG«B such that b|^ G Cj © Cj and b| - = s}. 

We will define a new tree model F = (T, {Se, e G E), (C^, v G V)), such that states in the 
same coset ofW vaSe are represented by a single "merged" state in Se- 
Let 

$ : e ( 05e I © 5e ^ F^ e I 05e I © Se/W 

be the mapping defined by 

^>((xi, z G /), (se, e 7^ e), s) = ((xj, i G /), (se, e 7^ e), s + TV). 

Define ^ = ^>(*B). It is clear from the definitions that = !B|^ = C, and that dim(^) < 
dim(«B). 

Consider now the tree model F = {T,lo, {Se, e G E), (C^, v G V)), where Se = for 
each e ^ E, and Cv = 53 1^ for each v ^ V. Note that Sg = Sg/W, and for e 7^ e, we have 
Se = ^\e = = '^e- AH States in Sg belonging to the same coset of W, say, s + W, are mapped 
to (or merged into) the single state s + W inSe- Further note that if v is not one of the two vertices 
incident with e, then Cv = 53 1^ = 53|^ = Cv At the two vertices that are incident with e, the local 
constraints are appropriately modified to take into account the state-merging at edge e. In any case, 
we have 

dim(5e) = dim(^|J < dim(Q3|J = dim(5e), for each e G E, (7) 

and 

dim(C^) = dim(^|„) < dim(53l„) = dim(C^), for each v e V. (8) 



We claim that F is an essential tree realization of C. To prove this claim, we must show that 
Q3(r)|g = for all e e E, and that 'B{T)\j = C, where «B(r) denotes the full behavior off. 
Note that we do not claim that 53(F) = *B; indeed, this may not be true. 



This restriction can be dropped by considering ess(r) instead; see Theorem l3.4l 
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It is easy to see that the inclusion C C *B(r)|j holds. Indeed, since F = {T,lo, (^B|g, e G 
-B), (!B|y, V € F)), it is evident that any b G 53 satisfies all the local constraints of F, and hence is 
in «B(r). Therefore, ^ C 53(r), and in particular, C = '^\j Q ^{T)\j. 

The reverse inclusion, 53 (F) [j^ C C, follows from part (a) of the lemma below. 

Lemma 3.1. Let h be a global configuration in 5B(r). Then, 

(a) b|j € C; and 

(b) b|g = if and only if h\j G Cj © Cj. 

We defer the proof of the lemma to Appendix iBl Lemma [37Tl a) shows that *B(r)|j C C, thus 
proving that F is a tree realization of C. It remains to show that F is essential, i.e., that iB(F)|g = 
*B|g for all e ^ E. This is shown by the following simple argument. We have already seen that 
^ C <B(r), and hence, C «B(r)|g for all e e E. On the other hand, at any e G E, ^(r)\^ 
is, by definition, a subspace of 5e = 53 |g. Hence, F is essential, thus proving our original claim, 
which we record in the following proposition. 

Proposition 3.2. The tree model F is an essential tree realization ofC. 

Let us call the process described above of obtaining F from F as the state-merging process at 
edge e. We use the notation F = mergeg(F) to denote this transformation. Our goal now is to show 
that, starting from an essential tree realization, if we apply the state-merging process at each edge 
of the underlying tree, then we always end up with a minimal realization. A proof of this assertion 
requires the following technical lemma, whose proof we also defer to Appendix iBl 

Lemma 3.3. Suppose that there exists e' (z E — {e} such that the full behavior, *B, o/F satisfies 
the following property: for b G 53, we have b[g, =0 if and only if h\j G Cj(e/) © C'jf^e')- Then, for 
any b G S(F), we also have h\^i = if and only if b|j G Cj(e/) © ^J^e')- 

We are now in a position to prove the main result of this section, which provides a construction 

of M.{C;T,uj) via state merging. 

Theorem 3.4. Let V be a tree realization of C that extends the tree decomposition (T, to). Let 
ei, 62, ... , e\E\ ^ listing of the edges ofT. Set Tq = ess(F), and for i = 1,2,..., \E\, set 
Ti = merge^. (Fj_i). Then, the minimal tree realization ^A{C■, T, to). 

Proof. Let 53 denote the full behavior of F (and hence, also of ess(F)), and for i = 1, 2, . . . , |£'|, 
let ^B(Fj) denote the full behavior of Fj. By Proposition 13. 2[ each Fj is an essential tree realization 
of C. 

By Lemma [3TTT b). for any b G 5B(Fj), we have b|g = if and only if h\j G Cj(ei) © Cj(e )• 
Furthermore, by Lemma 1331 for any j > i, 53 (Fj) satisfies the following property: 
for any b G S(Fj), we have bj^^ = if and only if b| ^ G Cj{ei) © '^j(e )• 
def 

In particular, 53* = 53(F|£;|) satisfies the following property for z = 1, 2, . . . , \E\: 

for any b G 53*, we have b|g. = if and only if h\j G Cj(e.) © Cj^^.^. 

Let us call the above property (P). Property (P) has two important consequences. Firstly, it implies 
that if b G 53* is such that b|j = 0, then = for all e G E. This means that the projection 
vr : 53* — > C defined by vr(b) = b|j is in fact an isomorphism. 

For the second consequence of (P), consider, for any e G -E, the homomorphism /3e : C — > 53* |g 
defined by /3e(c) = {^-^{c))]^. This map is well-defined since vr is an isomorphism. Property 
(P) is equivalent to the assertion that, for any e G i?, the kernel of /3e is precisely Cj(e) © Cj^g-j. 
Therefore, 53* |, ^ C/{Cj^,) © Cj(g)). 

Thus, for each e € E, state space 53* is isomorphic to S* defined in ([T]), and the map Pe 
is the canonical projection map s* given by It easily follows that for each v £ V, 5B*|^ is 
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isomorphic to C* defined in ©. Hence, r|^| = (T,a;,(<B*|g, e e S),(«B*|^, v e F)) is the 
minimal reaUzation (C ; T, w ) . □ 

Observe that at each step of the procedure outlined in Theorem 13.41 the dimensions of the state 
spaces and the local constraints do not increase. To make this precise, given tree models T' = 
{T,uj, (5^, e G E), (C;, V G V)) and T" = {T,io, {S'J, e G E), (C v G V)), let us say that 
r' ^ r" if dim(5^) < dim(5^') for all e e E, and dim(C;) < dim(C;') for all v eV. Then, for 
r and Fj, i = 0, 1, 2 . . . , \E\, as in the statement of Theorem 13.41 we have by virtue of (O and ([8]), 

^\E\ =^ ^\E\-i ^ . . . ^ Ti ^ To = ess(r) ^ r. 

Thus, we have that if F is any tree realization of C that extends the tree decomposition {T,uj), 
then A4{C;T,uj) ^ F. We record this strong property of minimal realizations as a corollary to 
Theorem 13.41 

Corollary 3.5. Let (T, to) be a tree decomposition of the index set of a code C, and let A4{C;T,uj) = 
{T,uj,{S*, e G E),(C*, V G V)) be the corresponding minimal tree realization of C. Then, 
for any tree realization, {T,uJ,{Se, e G E),{Cv, v G V)), of C that extends {T,uj), we have 
dim(5*) < dim(Se) for all e G E, and dim{C*) < dim{Cy) for all v G V. 

The procedure outlined in Theorem 13.41 does not translate to an efficient algorithm for the con- 
struction of A4{C;T,uj). This is because the state-merging procedure that creates Fj from Fj„i 
requires knowledge of the full behavior of Fj_i, which may not be easily determined. So, as a 
practical method for constructing A4{C; T, uj), given C and (T, to), we propose a novel construction 
that relies upon the code decomposition techniques of the next section. 

4. Code Decompositions 

In previous work lUTl . it was demonstrated that techniques from the decomposition theory of 
matroids Il25l . ll26l could be put to good use in a coding-theoretic setting. The decomposition theory 
in that work was presented in the context of binary linear codes. As we will now show, the basic 
elements of that theory can be easily extended to cover the case of nonbinary codes as well. The 
object of this exercise is not just to create a more general code decomposition theory, but as we 
will see in the next section, this decomposition theory ties in very nicely with the theory of tree 
realizations. 

Let Ci and C2 be linear codes over the finite field! Fg = GF(g), defined on the index sets Ii and 
I2, respectively. Let /1A/2 denote the symmetric difference, (/i U I2) — {h H I2), of the index 
sets. We will construct a code S(Ci, C2) with /i A/2 as its index set. For x = (xj, i G /i) G Ci and 
y = (j/i, z G 12) G C2, let X * y = (cj, i G /i U I2) be defined by 

{Xi for i £ Ii- I2 
yi for i e I2- h 
Xi-Vi foriG/in/2. 

Setting Ci * C2 = {x * y : x G Ci, y G C2}, we see that Ci -k C2 has Ii U I2 as its index set. 
We take S(Ci,C2) to be the cross-section (Ci ^C2)j^^j^- Note that when Ii (1 12 = 0, we have 

S(Ci,C2) = Ci*C2 = CieC2. 

For i = 1,2, let cj^-* and C^'^^ denote the projection Ci[/jn/2 cross-section {Ci)j^^j^, 

(v) (s) 

respectively. The codes Q and Q , for i = 1, 2, all have Ii n I2 as their index set. The dimension 

of S(Ci,C2) can be expressed in terms of the codes Ci, c|^^ and C^^\ i = 1,2, as stated in the 
following lemma. 



Up to this point, we did not need to specify the number of elements in the finite field over which we were working, but 
from now on, it will be useful for us to do so. 
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Proposition 4.1. For codes Ci , C2, we have 

dim(S(Ci, C2)) = dim(Ci) + dim(C2) - dim(cj"^ n 4'^) - dim(cj^^ + C^^^), 
where cf^^ + C^^^ = {x + y : x G C^f , y G C^'^ }. 

Proof. For a code C, and a subset J of its index set, the kernel of the projection map n : C —^ C\j 
is isomorphic to Cj, and hence, dim(Cj) = dim(C) — dim(C| j). Thus, taking C = Ci * C2, and 
J = /1A/2, we find that 

dim(S(Ci,C2)) = dim(Ci ★Cs) - dim((Ci ★C2)|^^n/2) = dim(Ci *C2) - dim(ci''^ + 4^^), 
since (Ci -k C2) |^^n/2 = ^1'^ + ^2^^- '^^ "^"^^ ^^^""^ dim(Ci * C2) = dim(Ci) + dim(C2) - 

dim(cf)n4'))'. 

Let C2 be a copy of C2 defined on an index set that is disjoint from Ii. For each y G C2, denote 
by y its copy in C2. Consider the homomorphism : Ci © C2 — > Ci *C2 defined by (/)(x, y) = x*y. 
Note that x*y = iff :>^\i^_j^ = y | = and ^\i^ni2~y\iini2 = ^- Equivalently, x*y = iff 

(s) (s) I 

■^l/in/2 ^ ^1 ' yl/in/2 ^ ^2 ' ^'^'^ ~ yl/in/2' follows that the kernel of (p is isomorphic 
to 

{z: zGCi^^zG^'^}. 

which is simply ^ n C^''^ 

Hence, dim(Ci*C2) = dim(Ci ©C2) - dim(ker(</>)) = dim((:i) + dim(C2) - dim(cf M cj'^), 
as desired. □ 

We will restrict our attention to a particular instance of the 8(^1,^2) construction, in which we 

(p) (s) 

require that the codes and Q ', i = 1, 2, take on a specific form. We need to introduce some 
notation first. For each positive integer r, set rrir = {q^' — l)/{q — 1), and fix an r x m,. matrix, 
which we denote by Dr, over ¥q, with the property that each pair of columns of Dr is linearly 
independent over F^. Note that Dr is a parity-check matrix for an [mr,mr — r] Hamming code 
over Fq (cf. ||27l § 3.3]). Let A^. denote the dual of this Hamming code, i.e., is the [nir, r] code 
over ¥g generated by Dj.. The code A^. is sometimes referred to as a simplex code. 

We take a moment to record an important property of the matrix Dj. that we will use later. The 
column vectors of Dj. form a maximal subset of F^ with the property that each pair of vectors 
from the subset is linearly independent over Fg. This is due to the fact that the number of distinct 
one-dimensional subspaces of F^ is precisely nir. Therefore, any (column) vector in F^ is a scalar 
multiple of some column of Dj.. 

Given an r > 0, suppose that the codes Ci and C2, defined on the index sets Ii and I2, respec- 
tively, are such that |/i n /2I = mj., and for i = 1, 2, we have c|^^ = A^ and c|*^ = {0}. In such 
a case, S(Ci, C2) is called the r-sum of Ci and C2, and is denoted by Ci C2. It is convenient to 
extend this definition to the case of r = as well: when |/i n/2| = 0, the 0-sum Ci ©0C2 is defined 
to be the direct sum Ci © C2. 

Example 4.1. Consider the case of codes defined over the binary field ¥2. Note that Ai = {0, 1}. 
Suppose that |/i fl /2I = X and that the coordinates of Ci are C2 are ordered so that the index 
common to Ii and I2 corresponds to the last coordinate of Ci and the first coordinate 0/C2. The 
conditions necessary for the 1-sum Ci ©1 C2 to be defined can then be stated as 

(PI) ... 01 not a codeword ofCi, and the last coordinate ofCi is not identically zero; 
(P2) W . . .0 is not a codeword 0/C2, and the first coordinate 0/C2 is not identically zero. 

The composite code S{Ci,C2) resulting from €1,62 that satisfy (PI), (P2) above was studied in 
III7I , where it was actually called a "2-sum ". 
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We would also like to point out that the specialization of our r-sum operation to the case r = 2 
was called "3-sum" in To add to the confusion, there was in fact an operation called "3- 

sum" defined in II17II , but that, in a certain sense, dualizes the 2-sum operation we have given in 
this paper 

iv) is) 

For r > 0, note that if C]^' and q ' {i = 1,2) are in the form needed to define an r-sum, then 

Cf^ + Cf' = Ar, and cj^-* n C^''^ = {0}. Therefore, as a corollary to Proposition HU we have the 
following result (which also applies trivially to the r = case). 

Corollary 4.2. For r > 0, ifCi, C2 are such that Ci 0r C2 can be defined, then 

dim(Ci ©r C2) = dim(Ci) + dim(C2) — r. 



An elementary property of direct sums {i.e., 0-sums) is that a code C is expressible as a direct 
sum of smaller codes if and only if there exists a partition (J, J) of the index set of C such that 
dim(C| j) + dim(C| j) — dim(C) = 0. This property extends beautifully to r-sums in general. 

Theorem 4.3. Let C be a linear code over Fg, defined on the index set I, and let r be a positive 
integer Then, the following statements are equivalent. 

(a) C = Ci®r C2 for some codes Ci, C2. 

(b) There exists a partition (J, J) of I, with min{| J|, | Jj} > r, such that 

dim(C|j) + dim(C|j) - dim(C) = r. 

Proof (a) =^ (b): See Appendix O 

(b) ^ (a): We give here a complete proof of this direction of the theorem, as it gives an explicit 
construction of codes Ci, C2 such that C = Ci (Br C2, given a partition ( J, J) as in (b). The proof 
generalizes ideas from similar constructions presented in [17 1. 

Let (J, J) be a partition of / such that dim(C| j) + dim(C| j) — dim(C) = r. Set n = \I\ and 
k = dim(C), and let G be a x n generator matrix for C. Without loss of generality, we may assume 
that the columns of G are ordered so that the first \J\ columns are indexed by the elements of J, 
and the rest by the elements of J. In the following exposition, we will often permute the columns of 
G to bring the matrix into some desired form. Whenever this is the case, it will be tacitly assumed 
that column indices migrate with the columns. 

Let G\j and G\j denote the restrictions of G to the columns indexed by the elements of J and J, 
respectively; thus, G = [G\j G\j]. Let rank(G[j) = ki and rank(Glj) = k2; by our assumption 
on ( J, J), we have we have ki + k2 = k + r. 

Bring G into reduced row-echelon form (rref) over ¥q. Permuting within the columns of G|j 
and within those of G\j if necessary, rref(G) may be assumed to be of the form 

' 4i A O B 
O O Ik-k, c 

where Ij, for j = ki,k2 —r, denotes the j x j identity matrix, A is a. ki x {\J\ — ki) matrix, i? is a 
ki X {\J\ — k + ki) matrix, C is a (A; — ki) x ([ J| — k + ki) matrix, and the O's denote all-zeros 
matrices of appropriate sizes. 

O B 

The fact that the submatrix ^ must have rank equal to rank(G| j) = k2 implies 

that B must have rank k2 — {k — ki) = r. Hence, B has r linearly independent rows, call them 



G 



(9) 



^The 2-sum and 3-sum operations defined in U7i imposed additional conditions on the lengths of the codes involved in 
the sum, which we have dropped here. 
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bi, . . . , hr, which form a basis of the row-space of B. Permuting the first ki rows of G if necessary, 
we may assume that bi, . . . , b^. constitute the first r rows of B. (Permuting these rows of G will 
also permute the rows of the /^^ matrix, but the effects of this can be negated by appropriately 
permuting the first ki columns of G.) Any row of B is uniquely expressible as a linear combination 
(over ¥g) of bi, . . . , b^. In particular, for i = 1, 2, . . . , fci, the ith row of B can be uniquely 
expressed as X]j=i ^'^^ some Oj j E F^. 

Let us denote by di, . . . , d^, the rows of the r x rur generator matrix, Dr, of the code A^. Let 
X be the ki x matrix such that for z = 1,2,..., ki, the ith row of X equals Yl^j=i 
where the aij's are such that the ith row of B is X]j=i ctij^j- Thus, the row-space of X is the 
span of di, . . . , d^., i.e., it is the code A^. To the columns of X, we assign indices from some set 
Ix disjoint from /. 

Now, define the fci x (| J| + rrir) matrix 

Gi=[h, A X], (10) 

allowing the submatrix [Ik-i A] to retain its column indices from G. Also, define the k x (\J\ +mr) 
matiix 

' X O B 



again allowing the submatrix 



G2 = 

O B 



(11) 



to retain its column indices from G. Thus, the index 



def ■ def ' — 

set of the columns of Gi is /i = J U Ix, while that of the columns of G2 is I2 = /x U J. 

Finally, for i = 1,2, let Cj denote the code over ¥q generated by Gj. The following facts about 

Ci and C2 may be verified: 

(i) dim(Ci) = rank(Gi) = ki,i = 1, 2. 

(ii) Ci (Br C2 can be defined, so that by Corollary 14.21 dim(Ci C2) = ki + k2 — r = k = 
dim(C). 

(iii) All rows of G are in Ci C2. Since G generates the same code as G (recall that column 
indices get permuted along with columns), we see that Ci (Br C2 contains all the codewords 
of C. 

We leave the details of the routine verification of the above facts to the reader. It only remains to 
point out that facts (ii) and (iii) above show that Ci (Br C2 = C, thus completing the proof of the 
implication (b) =^ (a). □ 

The procedure described in the above proof can be formalized into an algorithm that takes as 
input a k X n generator matrix G (over ¥g) for C, and a partition (J, J) of the index set of C, and 
produces as output generator matrices of two codes Ci and C2 (and their associated index sets) such 
that C = Ci (Br C2, where r = dim(C| j) + dim(C| j) — dim(C). The run-time complexity of this 
procedure is determined by the following: 

• an rref computation to find G as in (|9ll; this can be carried out in 0{k'^n) time, which is the 
run-time complexity of bringing akxn matrix to reduced row-echelon form via elementary 
row operations; 

• the computations required to identify a basis (bi , . . . , b^) of the row-space of the matrix B, 
and correspondingly the coefficients Uij; this could be done by computing the rref of B, 
which would also take 0{k'^n) time; 

• the computations needed to determine the ki x rrir matrix X; each row of the matrix requires 
0{rmr) computations, and there are ki = 0(| J|) rows, so the computation of X takes 
0{\J\rmr) = 0{\J\rq''') time. 

Therefore, the entire procedure can be carried out in 0{k'^n + \ J\rq^) time. It is worth noting that 
the run-time complexity of the procedure is polynomial in n, k and q, but exponential in r. 
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C2 





J 



J 



Figure 5 . A tree realization of an r-sum decomposition. 



5. A Construction of M{C; T, lo) via Code Decompositions 

The above procedure for determining an r-rum decomposition of a given code forms the basis of 
a new construction of minimal tree realizations that we present in this section. The key observation 
behind this construction is that if a code C has a partition (J, J) of its index set such that dim(C| j) + 
dim(C| j) — dim(C) = r, then C has an essential tree realization of the form depicted in Figure [51 
The tree in the figure consists of a single edge e = {vi,V2}, the state space Se is the code A^, 
and the local constraint codes at the two vertices are the codes Ci and C2 such that Ci (Br C2 = C. 
In fact, this is the minimal realization Ai{C;T,uj), for the tree T consisting of the single edge 
e = {^1,^2}, and the index map u) such that uj'^{vi) = J and uj^^{v2) = J- This is simply 
because dim(5e) = dim(Ar) = r, so by virtue of ([5]l, Se has the same dimension as the state space 
S* in the minimal realization A^(C; T, w). So, by the uniqueness of minimal tree realizations, the 
tree realization depicted in Figure [5]is M-{C;T,ul>). 

To summarize, if C is a code defined on the index set /, and (T, lu) is a tree decomposition of / 
such that T consists of the single edge e = {vi,V2}, then we may construct A4(C; T, lo) as follows. 
Set J = Lu^^{vi) and J = u>~^{v2), and compute r = dim(C| j) + dim(C| j) — dim(C). Assign an 
index set Ia that is disjoint from I to the code A,.. Use the procedure in the proof of Theorem 14.31 

to determine codes Ci and C2, defined on the respective index sets Ii = J U /a and I2 = J U /a, 
such that C = Ci (Br C2- For i = 1,2, assign Ci to be the local constraint code at vertex Vi, and 
assign A^ to be the state space at edge e. The resulting tree model (T, a;, A^., Ci, C2) is the minimal 
tree realization M{C;T,uj). 

Before describing how the construction may be extended to the case of trees with more than one 
edge, we deal with the trivial case of trees without any edges. If T is a tree consisting of a single 
vertex v, and no edges, then given any code C defined on some index set I, there is only one way of 
realizing C on T. This is the realization (T, iv, C^), where uj is the unique mapping lo : I —>■ {v}, 
and Cy is the code C itself. Of course, this is also the minimal realization M-{C;T,lo). 

At this point, we know how to construct A4{C; T, lo), for any code C, and any tree decomposition 
(T, co) such that T has at most one edge. From this, we can recursively construct M.{C\T, lo) for 
any C and any (T, lo), as we now describe. 

Suppose that we know how to construct M.{C\T,u}) for any C, and any {T,lo) such that T has at 
most r] — 1 edges, for some integer rj > 2. Let C be a code defined on the index set /, and let (T, lo) 
be a tree decomposition such that \E{T)\ = rj. Pick any e = {^1,^2} £ E{T), and as usual, let Tg 
and Te be the two components of T — e. We will assume that vi € V{Te) and V2 G V{Te). Let 
J(e) = LO'^{V{Te)) andJ(e) = uj-^{V{T^)). Compute 

r = dim(C|^(-)) + dim(C|j(.)) - dim(C), (12) 

which determines the code A,.. Assign A^ an index set /a that is disjoint from /. Use the procedure 
in the proof of Theorem 14.31 to determine codes Ci and C2, defined on the respective index sets 

h = J{e) U /a and h = J(e) U /a, such that C = Ci®r C2. 
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T, 



'1)2 



J(e) 



J(e) 



Figure 6 . Depiction of the manner in which the tree decompositions (Tg , cji ) and 

(Tg, UJ2) are obtained from (T, u). 



Now, define the index maps wi : Ii 



V{T^) and u;2 : /2 ^ ^(Te) as follows (see Figure ij: 

if i £ J(e) 
if i G /a 




if i e J(e) 
if i G /a 



(13) 
(14) 



Thus, {Tg, uji) and (Tg, ^2) are tree decompositions of the index sets of Ci and C2, respectively. As 
neither E{Te) nor E{Tg) contains the edge e, we have \E{Tg)\ < 77 - 1 and |^(Tg)| < r] - 1. 
Therefore, by our assumption, we know how to construct Ai{Ci;Tg, loi) and M.{C2; Te,u>2)- Let 

M{Ci;T,,ui) = (t,,u;i, {SP,eeE{T,)), {C^^\veVm)^, (15) 

M{C2;T,,i02) = (Te,cu2, {SP, e G E{T,)), {C^ , v G V{T,))^ . (16) 
Finally, set T* = (T,uj, {Se, e G E{T)), (C„, v G F(T))), where 



(17) 



and 



( 


if e 




< A^, 


if e 


= e 


c(2) 


if e 


G E{T,) 




if V 






if V 





(18) 



Figure |7] contains a depiction of T*. It is easy to see that F* is a tree realization of C. Indeed, 
A4{Ci;Tg,uJi) is a realization of Ci, and 7W(C2; T^, ^2) is a realization of C2, and hence (as should 
be clear from Figure |7]l, P* is a realization of Ci C2 = C. It is not immediately obvious that P* is 
actually Ad{C; T, to), but this is in fact true, as stated in the following proposition, a proof of which 
is given in Appendix iDl 

Proposition 5.1. P* is the minimal tree realization M.{C; T, lo). 
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J(e) J(e) 
Figure 7 . A depiction of the construction of T* from M{Ci;Te;uJi) and A< (C2 ; Tg , u;2 ) ■ 

In summary, we have the following recursive procedure for constructing A4{C;T,ijj), given a 
code C and a tree decomposition (T, lu). 

Procedure MIN_REALZN(C, T, uj) 

Input: A k X n generator matrix for a code C, and a tree decomposition {T,uj) of 
the index set of C. 

Output: A specification of the state spaces and the local constraints in the minimal 
realization M.{C;T,lo). 

Step Ml. If T consists of a single vertex, then return Ai{C; T, lu) = (T, to, C). 

Step M2. If T contains at least one edge, then choose an e G E{T). Let vi be the 
vertex of Tg incident with e, and let f 2 be the vertex of incident with e. 
(M2.1) Compute r = dim(C| + dim(C|j(^)) - dim(C). 

(M2.2) Determine A,., and assign it an index set /a disjoint from J(e) U J(e). 

(M2.3) Determine codes Ci and C2, with index sets Ii = J(e) U /a and I2 = 

J(e) U /a, respectively, such that C = Ci ©r C2. 

(M2.4) Determine the index maps uji and uj2 as in ([T3] ) and ([T4l ). 

Step M3. Determine A4(Ci; Tg, lji) by calling MIN_REALZN(Ci, Tg, wi); deter- 
mine M{C2:,Te.,uJ2) by calling MIN_REALZN(C2, Tg, a;2). We may assume that 
>l(Ci; Te, wi) and M{C2; Tg, ^2) are in the form given in ([TSl) and ([T6l ). 

StepM4. Return 7W(C;r, (J ) = (r,Lj, (5e, e G E{T)), (C^,, u G F(T))), where 
Se and Ct, are as defined in (fTTl ) and (fTSl ). 



A simplified version of the above procedure may be obtained by choosing, in Step M2, the edge 
e to be an edge incident with a leaf of T. Then, one of the two components of T — e, say, Tg, 
consists of a single vertex, so that the call to MIN_REALZN(C2; Tg, 0^2) may be avoided, as it would 
simply return (Tg, 0^2, C2). We will use this modification of the procedure to give an estimate of its 
run-time complexity. 

Let n denote the length of C, let k = dim(C), and let E = E{T). Also, define 

^max = ma^ dim(C|j(g)) + dim(C|j(g)) - dim(C). (19) 

Observe that, as a result of the modification suggested above, in the determination of M{C; T, lo), 
the procedure MIN_REALZN gets called [£^1 times, once for each edge e E E. The run-time com- 
plexity of any particular run of MIN_REALZN is determined by the computations in Step M2. In 
the ith run, the procedure acts upon some code C^^^ of length rii and dimension ki, and in Step M2, 
it computes an rj, a code A^.. with index set , and a code c[^\ Via Lemma [DTT] we have that 
i^i < r-jnax- We bound ki and Ui as follows. Note that C^^ is the code C^*^^'' that the (i + l)th 
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run of the procedure takes as input. Thus, we have A;j_|_i < ki, and rij+i < rii + |. Since 

ki = k, ni = n, and \ = {q"^' — l)/{q — 1) < g''™, we have, for i = 1, 2, . . . , \E\, h < k 
and rii < n + {i — l)q^'^'^^. Now, by the estimate given in Section |4] of the run-time complexity 
of the r-sum decomposition procedure, we see that the ith run of Step M2 of MIN_REALZN takes 
0{kfni + riiriq"^^ ) time. Hence the overall run-time complexity of MIN_REALZN may be estimated 

to be ^If ^ Oik^m + mnq''^). This expression can be simplified by observing that 

\E\ \E\ 
^(/c^n, + nir^/O < (k^ + r^^^q''^^^) ^ rii 

i=l i=\ 

\E\ 

< {k^ + r-maxg'''"^") ^[n+{i- l)q''^^''] 

i=l 

= (k^ + rmaxg"'"^'') [n\E\ + (1/2)|E|(|^| - 1)/—] . 

It follows that MIN_REALZNruns in 0{{k'^ + rmax9''™)('^|-£^| + l-E^pJ^max time. Note that 
this is polynomial in n, k, q and \E\, but exponential in rmax- 



6. Complexity Measures 

6.1. Complexity Measures for Codes. As observed in Q, any graphical realization of a code 
specifies an associated decoding algorithm, namely, the sum-product algorithm. The sum-product 
algorithm specified by a tree realization, F = {T,uj, {Se, e £ E), {Cv, v G F)), of a code C 
provides an exact implementation of ML decoding for C. A reasonable initial estimate of the com- 
putational complexity of the sum-product algorithm on F is provided by the constraint complexity 
of F, which is defined as max^gy dim(Ct,). As implied by Corollary 13. 5[ given a tree decom- 
position {T,Lo) of the index set of C, the minimal realization M{C]T,u>) has the least constraint 
complexity among all tree realizations of C that extend (T, u)). Let k{C;T,u}) denote the constraint 
complexity of M{C; T, uj). Note that, by Q, 

k{C;T,uj) = max dim(C) — dim(Cj(e)) 1 . (20) 

''^^ V eeE(^;) / 

Thus, k{C;T,uj) is a measure of the complexity of implementing ML decoding for C as a sum- 
product algorithm on Ai{C;T,uj). 

Let us now define the treewidth of the code C to be 

hi(C) = min k(C;T,lj), (21) 

(T,a;) 

where the minimum is taken over all tree decompositions (T, lo) of the index set of C. The treewidth 
of a code is an indicator of how small the computational complexity of an ML decoding algorithm 
for C can be. The notion of treewidth (i.e., minimal constraint complexity) of a code was first 
introduced by Forney fS]. A related notion, called minimal tree complexity, was defined and studied 
by Halford and Chugg |9]. Treewidth, as defined in (|2TI ). is an upper bound on the minimal tree 
complexity measure of Halford and Chugg. 

A tree is called cubic if all its internal nodes have degree 3. Forney fF] showed that the minimum 
in (|2TI) is always achieved by a tree decomposition (T, lo) in which T is a cubic tree, and lj is a 
bijectioijl between the index set of C and the set of leaves of T. Let Q(C) denote the set of all tree 



Forney fSl only explicitly states that the minimizing (T, lj) may be taken to be such that T is a cubic tree and is a 
surjective map onto the leaves of T. However, the symbol-splitting argument in Section V.F of his paper actually implies 
that Lj in the minimizing tree decomposition may be taken to be one-to-one as well. 
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decompositions (T, lu) in which T is cubic and uj maps the index set of C bijectively onto the set of 
leaves of T. We may then re-write (|2TI) as 

k(C) = min k(C;T,lo). (22) 

{T,uj)€Q{C) 

An alternate measure of code complexity may be obtained from the notion of state complexity of 
a tree realization F, which is the largest dimension of a state space in F. Thus, by virtue of ^ and 
([5]), the state complexity of a minimal realization Ai{C; T, lu) is given by 

a{C;T,u)) = max dim(C) — dim(Cj(e)) — dim{Cj^^-^) 

= max dim(C|j(g)) + dim{C\j^^-^) - dim(C). (23) 

We then define, in analogy with (l22l) . 

a(C) = min a(C;T,uj). (24) 

{T,uj)eQ{C) 

Note that the minimum in the above definition is taken over tree decompositions in Q(C) only. It 
must be emphasized that a{C), as defined in ((24l) . need not be the same as the least (t(C; T, w) over 
all tree decompositions (T, lu) of the index set of C. 

A notion analogous to (7(C) is known as branchwidth in the mati^oid theory literature; see e.g., 
115] ■ In keeping with that nomenclature, we will call a{C) the branchwidth of the code C. Branch- 
width and treewidth are very closely related, as shown by the following result, which can be ob- 
tained in a straightforward manner from the bounds in Lemma 1231 

Proposition 6.1 (HU, Theorem 4.2). Given a code C, if{T, cu) G Q{C), then 

a{C;T,Lu) < k{C;T,lo) < 2a{C;T,Lu). 

Hence, a{C) < k{C) < 2a{C). 

The notions of state and constraint complexity have been studied extensively in the context of 
conventional trellis realizations of a code; see e.g., [281. Recall from Example 12.21 that a conven- 
tional trellis realization of a code is a tree realization that extends a tree decomposition {T,lu) in 
which T is a simple path and a; is a bijection between the index set of C and the vertices of T. 
This special case of a tree decomposition is referred to as a path decomposition. Specifically, a path 
decomposition of a code C defined on the index set / is a pair (T, lu), where T is a simple path on 
|/| vertices, and lu : I V{T) is a bijection. Let V{C) denote the set of all path decompositions 
of C. We then define 

'^treiiis(C) = min k{C;T,uj) (25) 
iT,u))eV{c) 

and 

o-treiiis(C) = min a{C;T,u;). (26) 

{T,io)£r{c) 

It is well-known, and indeed readily follows from Lemma 1231 that (Ttieiiis(C) < Ktreiiis(C) < 
Ctrellis (C) + L 

It is clear from (|2TI ) and (l25l ) that k{C) < Ktreiiis(C). Forney fF] asked the question of whether 
K,{C) could be significantly smaller than Ktreiiis(C). He conjectured that either Ktreiiis(C) — k{C) < 1 
for all codes C, or Ktreiiis(C) — k{C) is unbounded. We show here that it is in fact the latter that is 
true. To do so, we need to introduce some new concepts. 



ON MINIMAL TREE REALIZATIONS OF LINEAR CODES 



19 



6.2. Complexity Measures for Graphs. In their fundamental work on graph minors Il24l . Robert- 
son and Seymour introduced two notions of complexity of graphs, namely, treewidth and pathwidth. 
These notions have proved to be invaluable tools with many applications in graph theory and theo- 
retical computer science. An overview of such applications can be found, for example, in |6|. We 
will define the notions of treewidth and pathwidth of a graph in this subsection, and subsequently, 
relate them to the complexity measures k{C) and Ktreiiis(C) defined above for codes. 

Let ^ be a graph with vertex setV{Q) and edge set E{Q). The graph may contain self-loops and 
parallel edges. A tree decomposition of C/ is a pair (T, f3), where T is a tree, and /? : V{T) — > 2^^^^ 
is a mapping that satisfies the following: 

(Tl) UeF(T)/3(^) = m; 

(T2) for each pair of adjacent vertices u,v G V{Q), we have {u, v} C (3[x) for some x € V{T); 
and 

(T3) for each pair of vertices x, z G V{T), if y G ^iT) is any vertex on the unique path between 
X and z, then /?(x) n C p{y). 

It may be helpful to point out that (T3) above is equivalent to the following: 

(T3') for each v G V{Q), the subgraph of T induced by {x G V{T) : v G P{x)} is a (connected) 
subtree of T. 

A reader familiar with the notion of "junction trees" (see e.g., Q) will recognize a tree decomposi- 
tion of Q to be a junction tree. 

The width of a tree decomposition (T, /?) as above is defined to be max^^Y(^j') ~ 1- The 

treewidth of G, which we denote by k{Q), is the minimum among the widths of all its tree decom- 
positions. Note that if G has at least one edge, then, because of (T2), any tree decomposition of Q 
must have width at least one. Thus, for any graph Q with > 1, we have k{G) > 1- 

Example 6.1. For any tree T with at least two vertices, we have k{T) = 1. This can be seen as 
follows. Fix a vertex r G V{T). Define a mapping f3 : V{T) 2^^^^^ as follows: (3{r) = {r}, and 
for X ^ r, (3{x) = {x, y}, where {x, y} is the first edge on the unique path from x to r. It is easily 
verified that {T,P) is a tree decomposition ofT. Since this tree decomposition has width one, it 
follows that k{T) = 1. 

If (T, (3) is a tree decomposition in which T is a simple path, then (T, f3) is called a path de- 
composition. The minimum among the widths of all the path decompositions of G is called the 
pathwidth of G, which we denote by Kpath(^)- It is evident that k{G) < Kpath(^). 

Analogous to the situation of Example l6.1[ a simple path has pathwidth one. However, trees may 
have arbitrarily large pathwidth. The following example is due to Robertson and Seymour |[23]| . 

Example 6.2. Let Yi be the complete bipartite graph Ki ^,. For i > 2, we inductively define Yi 
by taking a copy ofYi^i, and to each leaf v of this graph, adding two new vertices adjacent to v. 
Figure^shows the trees Yi, Y2 and Y^. The pathwidth ofYi, i > 1, is [^(i + 1)] ll23l . 

Thus, for trees T, the difference Kpath(r) — k(T) can be arbitrarily large. We will use this fact 
to construct codes C for which Ktreiiis(C) — n{C) is arbitrarily large. 

We remark that the problem of determining the treewidth or pathwidth of a graph is known to 
be NP-hard 121,1161. As we will see a little later, this implies that the problem of determining the 
treewidth of a code, or its trellis counterpart, is also NP-hard. 

6.3. Relating the Complexity Measures for Codes and Graphs. Let F be an arbitrary finite field. 
To any given graph G, we will associate a code C[G] over F as follows. Let D{G) be any directed 
graph obtained by arbitrarily assigning orientations to the edges of G, and let ^z)(g) be the vertex- 
edge incidence matrix of D{G). This is the x matrix whose rows and columns are 
indexed by the vertices and directed edges, respectively, of D{G), and whose (i,i)th entry, aj j, is 
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Figure 8. The trees Yi, Y2 and Y3. 




g G' G 



Figure 9. Construction of Q' and Q from Q. 

determined as follows: 

{1 if vertex i is the tail of non-loop edge j 
— 1 if vertex i is the head of non-loop edge j 
otherwise. 

The code C[Q] is defined to be the linear code over F generated by the matrix ^z)(g). When F is the 
binary field, the code C[Q] is the cut-set code of Q, i.e., the dual of the cycle code of Q ifTOl . 

The following fundamental result that relates the treewidths of the graph Q and the code C[Q] is 
due to Hhneny and Whittled [161. 

Theorem 6.2 ( |[T6l . Theorem 3.2). If Q is a graph with at least one edge, then k{Q) = k{C[Q]). 

Since determining the treewidth of a graph is NP-hard, it immediately follows from the above 
theorem that the problem of determining the treewidth of a code (over any fixed finite field) is also 
NP-hard. We remark that the problem of determining the branchwidth of a code is also NP-hard. 
This follows from a result [ 1 1] that relates the branchwidth of the code C[Q] to the branchwidth of 
the graph G, the latter being a notion we have not defined in this paper. 

Unfortunately, it is not true that Kpath(^) = ^trellis As an example, consider the code C[T] 
over the binary field, for an arbitrary tree T. It is not hard to see that C[T] = {0, which, 
being the direct sum of multiple copies of {0, 1}, has ^treiiis (C [T] ) = 1. But as we have already 
noted, trees can have arbitrarily large pathwidth. 

We get around this problem by means of a suitable transformation of graphs. Given a graph Q, 
let Q' be a graph defined on the same vertex set as Q, having the following properties (see Figure|9ll: 

• Q' is loopless; 

• a pair of distinct vertices is adjacent in Q' iff it is adjacent in Q; and 

• in Q', there are exactly two edges between each pair of adjacent vertices. 



The results in II6I are stated in matroid-theoretic language. The vocabulary necessary to translate the language of 
matroid theory into that of coding theory can be found, for example, in IT 81. 
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Define Q to be the graph obtained by adding an extra vertex, x, to Q' , along with a pair of parallel 
edges from x to each v e V{Q') (see Figure|9]). It is easy to see that Q is constructible directly from 
ginO(|y(^)|2)time. 

The following result was used in |18J to show that the problem of determining cTtieiiis(C) for an 
arbitrary code C (over any fixed finite field) is NP-hard. 

Theorem 6.3 ( 11181 . Proposition 3.1). If Q is the graph constructed from a given graph Q as de- 
scribed above, then atreius (C [G] ) = {Q) + 1- 

Since o"treiiis(C) is always within one of Ktreiiis(C), the above theorem implies that 

«^path(^) + 1 < I^Xvt\\i^{C[Q]) < Kp^thiG) + 2. (27) 

While this falls short of establishing the NP-hardness of computing KtreiUs (C) for an arbitrary code C, 
it is certainly enough to provide us with the desired example of codes C for which Ktreiiis (C) — k{C) 
is arbitrarily large. We just need to make one more observation: k{Q) = k{Q) + 1. The proof 
of this fact, which is along the lines of the proof of Lemma 3.5 in ifTSl . is left to the reader as a 
straightforward exercise. We can now prove the following corollary to Theorems 16. 21 and 1 6.31 

Corollary 6.4. Over any finite field ¥, there exists a family of codes Cj, z G 1, 2, . . ., such that 

lim KtrellhiCi) - K.{Ci) = OO. 



Proof. Let Yi, i = 1,2, . . ., be the family of trees defined in Example 16.21 Define d = C[Yi\, 
where Yi refers to the graph obtained from Yi by the transformation depicted in Figure |9l Note that 
K(Yi) = K{Yi) + 1 = 2, since K{Yi) = 1, as shown in Example 16. II Thus, on the one hand, from 
Theorem 16.21 we have K{Ci) = K{Yi) = 2. And on the other hand, from (1271) and Example I6.2[ we 

have Ktrellis(Ci) > Kpath(il) + 1 = + 3)]. □ 

Using standard facts known about the incidence matrix ^D(g) for a graph Q (see, for example, 
II22I Chapter 5]), it may be verified that the codes Ci, i > 1, constructed in the above proof are 

[rii , ki , di] codes, where 

n^ = \E{Yi)\ = 12(2^-1) + 2, 

ki = 1^(1^)1-1 = 3(2'-l) + l, 

di = size of the smallest cut-set in 1^ = 4. 

Note that Ktreiiis (Ci) — K{Ci) grows as 0(log rii). We conjecture that this is in fact the maximal rate 
of growth of the difference Ktreiiis (Ci) — K,{Ci) for any code family Ci. 

Conjecture 6.1. IfCi, i > 1, is any sequence of codes over¥, then 

l^trellis{Ci) — K{Ci) 

limsup < OO, 

i log Hi 

where rii denotes the length of the code Ci. 

The codes Ci constructed in the proof of Corollary 16.41 all have treewidth equal to two. Issues 
related to families of codes whose treewidth is bounded by a constant are discussed next. 

6.4. Codes of Bounded Complexity. Many NP-hard combinatorial problems on graphs are known 
to be solvable in polynomial (often, linear) time when restricted to graphs of bounded treewidth 
Q,!!!!. In this subsection, we will see that the same general principle applies to problems pertaining 
to codes as well. 

Let = GF{q) be a fixed finite field. Given an integer t > 0, denote by TW(t) (resp. BW(t)) 
the family of all codes over Fg of treewidth (resp. branchwidth) at most t. Thus, a family of codes 
over ¥q is said to have bounded treewidth (resp. branchwidth) if C TW(t) (resp. € C BW(t)) 
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for some integer t. Note that by Proposition lOl BW([t/2j) C TW(t) C B\N{t), and so, a code 
family € has bounded treewidth if and only if it has bounded branchwidth. 

A fundamental result of coding theory |4| states that the problem of ML decoding is NP-hard 
for an arbitrary family of codes. However, we will now show that this problem becomes solvable 
in linear time for any code family of bounded treewidth. So, consider a code family C C T\N{t), 
where t is a. fixed integer, and pick an arbitrary C G Let n denote the length of C. By definition, 
C has a minimal realization Ai{C; T,uj) with constraint complexity at most t. Moreover, by (l22l ). 
(T, Lu) can be chosen to be in Q{C), i.e., it may be chosen so that T is a cubic tree, and uj maps the 
index set of C bijectively onto the leaves of T. In particular, the number of leaves of T equals the 
cardinality, n, of the index set of C. 

Now, recall that ML decoding of C may be implemented as a sum-product algorithm on any 
tree realization of C, and in particular, on A4{C; T, to). The computational complexity of the sum- 
product algorithm on ^A{C■,T,uJ) is determined by the computations that take place at the internal 
nodes of T. By an estimate of Forney |lL Theorem 5.2], the number of computations at the internal 
node V G V(T) is of the order of — 2)g^™('^''\ where 6^ is the degree of v in T. Since 
T is cubic, 6y = 3, and since the constraint complexity of Ai{C;T,uj) is at most t, we have 
dim(C*) < t. Hence, the number of computations performed by the sum-product algorithm at any 
internal node of T is bounded by Sg*, which is a constant. Now, T is a cubic tree on n leaves, so it 
has at most n — 2 internal nodes. It follows that the computational complexity of the sum-product 
algorithm on A4{C; T, to) is 0(n), the constant in the O-notation being proportional to 3g*. Thus, 
there is a linear-time implementation of ML decoding for any C € G!. 

A question that naturally arises in this context is that of how hard it is to explicitly determine the 
minimal tree realization required for linear-time implementation of ML decoding. Note that this is 
not exactly a decoding complexity issue, since the determination of a suitable A4{C; T, lu) may be 
done "off-line" for each C € GT. 

An explicit determination of 7V4(C;T, involves finding a tree decomposition {T,lo) € Q{C) 
such that k{C; T, u) < t, and the specification of the state spaces and the local constraint codes of 
A4{C;T,Lu). Given a (T, lu) € Q{C) satisfying k{C;T,lo) < t, the state spaces and local constraint 
codes of A4{C; T, lu) may be determined by the MIN_REALZN procedure of Section|5] An estimate 
of the computational complexity of this procedure was given in that section, in terms of the length 
and dimension of C, the number of edges in T, and the quantity rmax defined in ([T9l) . Comparing 
(fT9l ) with (l23l) . we see that rmax is simply a{C; T, uj), which by Proposition 16. 1 [ is bounded from 
above by t. The number of edges of T is |F(T)| — 1, and V{T) consists of n leaves and at most 
n — 2 internal nodes, n being the length of C. Therefore, by the estimate of the computational 
complexity of MIN_REALZN given in Section HI for an [n, k] code C £ <t, and a {T,uj) G Q{C) 
such that k{C; T, uj) < t, the minimal realization A4{C; T, uj) may be constructed in 0{k'^v?') time. 
Note that t appears in the exponent of the constant implicit in the O-notation. 

This leaves us with the problem of finding, for a given code C G TW(t), a tree decomposition 
{T,uj) G Q(C) such that k{C; T,uj) < t. Unfortunately, there appears to be no efficient algorithm 
known for solving this problem. However, reasonably good algorithms do exist for solving a closely 
related problem: given a code C G BW(t), find a tree decomposition (T, w) G Q{C) such that 
a{C; T, uj) < t. Several polynomial-time algorithms for solving this problem are given in |T4], the 
most efficient of these being an algorithm that runs in 0{n^) timeO, n being the length of C. Now, 
by Proposition iH any C G TW(t) is also in B\N{t), and furthermore, k{C;T,uj) < 2a{C;T,uj). 
Therefore, the algorithms of fT4 1l find, for a given code C G TW(t), a tree decomposition (T, uj) G 
Q(C) such that k{C\ T, uj) < 2t. This is sufficient for our purposes, as the computational complexity 
of the sum-product algorithm on the resulting A4{C;T,uj) would still be 0{n), except that the 
constant in the O-notation would now be proportional to 3g^*. 



As usual, the constant hidden in the O-notation depends exponentially on t. 



ON MINIMAL TREE REALIZATIONS OF LINEAR CODES 



23 



While code families of bounded treewidth have the desirable property of having linear decoding 
complexity, it is very likely that they do not have good error-correcting properties. We give an 
argument to support the plausibility of this statement. Recall from coding theory that a code family 
<t is called asymptotically good if there exists a sequence of [n^, fcj, di] codes Ci S with limj ni = 
oo, such that liminfj fcj/rij and liminfj di/rii are both strictly positive. The code family Ci, i >1, 
from the proof of Corollary 16.41 has bounded treewidth, but is not asymptotically good: ki/rii — > 
1/4, but dj/nj — > 0, as i ^ oo. 

It is known that if a code family <t has bounded trellis complexity, i.e., if there exists an integer t 
such that (Ttreiiis(C) < t for all C G then C is not asymptotically good. This is a consequence of 
the following bound for an [n,k,d] code C [|2U : 

fTtrellis(C) > (28) 

n 

Since crtreiiis(C) < Ktreiiis(C), the above is also a lower bound on Ktreiiis(C). 

Now, suppose that Ci, i > 1, is a sequence of [rii, ki,di] codes of bounded treewidth, so that there 
exists some t > such that for all i, K{Ci) < t. Hence, from (1281 ). we have Ktreiiis(Cj) — K{Ci) > 
^ {di — 1) — t. Therefore, assuming the validity of Conjecture 16. 11 we have 

limsup < oo. 

i log Hi 

Since t is a constant, this imphes that 

limsup — < oo, 

i Hi log Ui 

from which we infer that at least one of lim inf j ki / rii and lim inf , di / Ui must be zero. 

Observe that the final conclusion of the above argument can also be reached if we only assume 
that K{Ci) grows at most logarithmically with n^. We formalize this as a conjecture. 

Conjecture 6.2. Let d, i > 1, be any sequence of [rii, ki, di] codes such that limsupj i}^^. < oo. 
Then, lim inf j ki / rii cmd lim inf j di /rii cannot both be strictly positive. In particular, for any t > 0, 
the code family TW{t) is not asymptotically good. 

We wrap up our discussion on complexity measures for codes by elaborating on a comment we 
made at the beginning of this subsection, in which we implied that hard coding-theoretic problems 
often become polynomial-time solvable when restricted to codes of bounded complexity. We saw 
earlier several examples of algorithms that, given a code C € TW(t), solve some problem in time 
polynomial in the length of C. In each of these cases, the computational complexity of the algorithm 
displayed an exponential dependence on the parameter t. But since t was a fixed constant, this 
exponential dependence could be absorbed into the constant hidden in the "big-0" estimate of 
the complexity. Thus, fixing the parameter t allowed a potentially intractable coding-theoretic 
problem to become tractable. Problems that may be hard in general, but which become solvable 
in polynomial time when one of the parameters of the problem is fixed, are called fixed-parameter 
tractable. We noted previously that the problems of computing the treewidth and branchwidth of 
a code are NP-hard. It should come as no surprise that these problems are in fact fixed-parameter 
tractable. Hlineny (T2\ gives an O(n^) algorithm that, for a fixed integer t, determines whether or 
not a given length-n code is in BW(t). From this, one can also prove the existence of an 0{n'^) 
algorithm for deciding membership of a given length-n code in TW(t) fT3l . 

7. Concluding Remarks 

Perhaps the most significant problem that remains open in the context of minimal tree realiza- 
tions of codes is the resolution of Conjecture 16.21 which proposes that codes of bounded treewidth 
cannot be asymptotically good. It may be possible to resolve this by deriving a lower bound on 
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treewidth along the lines of the bound in ( |28l ). Another possibility is an inductive approach using 
code decompositions. 

However, an open problem of far greater significance is the development of a general theory of 
minimal realizations of codes on graphs with cycles. At present, such a theory only exists for the 
case of realizations of codes on graphs consisting of a single cycle, i.e., tail-biting trellis realizations 
1201. This simplest case of graphs with cycles is already more difficult to study than the cycle-free 
case — for example, there can be several non-equivalent definitions of minimality in the context of 
tail-biting trellis realizations. The challenge posed by graphs with more complex cycle structures 
can only be greater. 

Appendix A. Proofs of Lemmas UTTI and \272\ 

Proof of Lemma \2.1\ Consider an arbitrary e G E. An arbitrary global configuration b may be 
written in the form (b| j^g^, b|g(^^^^, b|g, ^\E(Te)'''^hie))- Now, suppose that b is such that = 0, 
i.e., b = (b| b|^(^-p^^, 0, bl^;^^^^, b| j^^^. Observe that the global configurations 

b' = (b|j(,),b|^(^^), 0,0,0) and b" = (0, 0, 0, b|^(^^), b|7(g)) 

also satisfy all local constraints (since G 5S|^ for each v G V), and hence are in *B. There- 
fore, (b|j(gj,0) = h'\j G C, and so by definition of Cj(e), we have h\jf^^-^ G Cj(e). Similarly, 
(OMjie)) = e C, so that b|j(g) G Cj(g). Hence, b|, = {h\j^^^,h\jie)) ^ Cj(e) C-j^.y □ 

Proof of Lemma \T2\ For any tree model (essential or not), we have, by definition, !B|^ C C„ for 
all V & V. So we need only show the reverse inclusion in the case when F = (T, w, {Se, e G 
E), {Cy, V £ V)) is an essential tree model. 

Pick an arbitrary v G V. Let ei, 62, • • • , be the edges of T incident with V. For i = 
1,2, ... ,6, let Tj denote the component of T — that does not include v. Set Fi = E(Ti), and 
Ji = io^^{V{T.i)). We will write an arbitrary configuration b G *B as 

(bL-i(„), (b[g^,b|^^, b|^J, . . . , (b|g^, bl^^.,b|^j) . 

Consider any (cq, ci, . . . , c^) G where cq G F'^ and Cj G Se^ for z = I, ... ,6. As 
the tree model F is essential, we have & = 5BL for all i. In particular, Cj G iBl , so that there 

exists b^*) G 53 such that b(*)|g, = Cj. As b^*) is in 53, its "sub-configuration" (cj, b^*) l^;'., b^*) |j.) 
satisfies the local constraints of F at all vertices in V{Ti). Hence, 

b = (co, (ci,bW|^^,b«j^J,...,(c,,bW|^^,b(^)|^j) 

satisfies the local constraints of F at all vertices in IJi=i ^(^i)- Now, v is the only vertex of T 
that is not in |jf=i ^(^i)- ^^t, by construction, b|^ = (co,ci, . . . ,cs) G Cy, and so, b also 
satisfies the local constraint at v. Thus, b satisfies all local constraints of F, so that b G 53. Hence, 
(cq, ci, . . . , C5) = b|^ is in 53|^, which proves the lemma. □ 

Appendix B. Proofs of Lemmas [311 and [331 

Proof of Lemma l3J\ For simplicity of notation, let F denote the edge set of the subtree Tg, and let 
F denote that of the subtree Tg. Note that FUF = E(T) — e. Throughout this proof, we will write 
an arbitrary global configuration b, belonging to 53 or ^B(F), in the form (b| j, b|^, b| h\-p, h\j). 

Consider any b = (b| j, b|p,, b|g, b|-p, b| j) G ^B(F). Let £ and r be the two vertices incident 
with the edge e in T. We assume that i G V{Tg) and r G V{Tg), as depicted in Figure [TO] We 
write the local configuration b|^ as (b|^(^)_-, b|^_i(^), b|.), and b|^ as {h\g,h\^-i^j,yh\^^^^_g). 

Suppose first that h\g = 0; note that the zero element of Se (= Sg/W) is W. By definition of F, 
b|^ G = <&(*B)|£. Hence, there exists (b|^(-£)_^, bj^-i^^^, w) G iBj^, for some w £ W. Now, 
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m J{e) 
Figure 10. A depiction of the two subtrees and Tg connected by the edge e. 

(b|j,b|j7) (being a "sub-configuration" of b) satisfies the local constraints of F at all vertices in 
V{Te) — {£}. But these local constraints are of the form *B I which, for u € y(Te) — {£}, is identical 
to *B^. Therefore, the sub-configuration (b|j, b|p) satisfies the local constraints of F at all vertices 
in V{Tg) — {£}. It follows that (b| j, h\p, w) satisfies the local constraints of F at all vertices in 
V{Tg,), including £. By a similar argument, there exists aw' € W such that (w', b|-p, b| j) satisfies 
the local constraints of F at all vertices in V{Tg). 

Now, by definition of W, there exist b and b' in *B, such that b = (b| j, b|j:^, w, b|-p, b| j), 
b' = {h'\j,h'\p,w',h'\-p,h'\j), and (b| j, b|j), (b'j b'|j) G Cj © C j. Note, in particular, that 
the sub-configuration (w, b|-p, h\j) of b satisfies the local constraints of F at all vertices in V{Te). 
Therefore, the global configuration g = (b| j, b|^, w, b|;p, b| j) satisfies the local constraints of 
F at all vertices in T, and hence is in the full behavior, iB, of F. A similar argument shows that 
g' = (b'[j, b'[^, w', b|;p, b| j) is also in *B. 

As OS is a vector space, it must also contain 

b-g = (b|^-b|^,b|^-b|^,0,0,0) 

and 

b' - g' = (0, 0, 0, b V - b|5?, b' \j - h\j) . 

Since F is a tree realization of C, we have 53|j = C. In particular, (b| j — b| j, 0) = (b — g)\j € C, 
and similarly, (0, b'|j - b|j) = (b' - g')|j G C. Hence, h\j - h\j G Cj and b'|j - b|j G Cj. 
However, b and b' were chosen so that b| j G Cj and h'\j G Cj. Thus, we also have b| j G Cj and 
b|j G Cj. This finally yields h\j = (b| j, b| j) G Cj © Cj, thus proving one direction of part (b) of 
the lemma. 

We will next show that if b| - / 0, then b/ G C but b/ ^ Cj © Cj. This will prove both part (a) 
and the reverse direction of part (b). 

So, suppose that b|g = s 7^ 0. Thus, s is some coset of W in S^, but is not W itself. Pick some 
s G s. As 5e = ?B|-, there exists some b G *B such that b|- = s. Observe that h\j G 53|j = C, but 
since s Gs ^ W,h\j ^ Cj (B Cj. 

Define b = ^>(b), so that b G 53. Furthermore, b|g = s, and h\j (= h\j) is in C but not in 
Cj © Cj. We have already noted (prior to the statement of Lemma [XT] ) that *B C 53(F). Therefore, 
b G *B(r), and since *B(r) is a vector space, b - b G *B(r). 

However, (b — h)\- = s—s = 0, and as we showed above, this implies that (b — h)\j G CjffiCj. 

Since b| ^ is in C but not in Cj © Cj, we find that h\j G C, but h\j ^Cj® Cj. 

The proof of the lemma is now complete. □ 

Proof of Lemma [Ol As F is a tree realization of C, Lemma |2?T] shows that for any b G 53(F), we 
have b|g, = only if h\j G Cj(e/) © Cj^^,y Thus, we need only prove the converse. 
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Suppose that b G *B(r) is such that (b| blj^^,)) e Cj(e/) Cj^^,y but b|g, / 0. Now, 

b|g, G Se' = ^B|g; = 5B|g/, the last equahty being a consequence of the fact that e' / e. Therefore, 
there exists a b € 53 such that h\^, = h\^,. Note that, by the hypothesis of the lemma, h\j ^ 

Set b = ^{h), so that b G *B C !B(r). Observe that h\j = h\j, and since e' ^ e, we also have 
b|g, = b|g,. Thus, b|g, = b|g, and h\j ^ Cj(e/) © ^l{e')- '^^^^ b — b G 5B(r), with 

(b — b)|g, = 0, and (b — b)|g, ^ Cj(e') © ^7(e')- ^^^^ contradiction of Lemma 12. 1 1 proves that 
there exists no b G *B(r) such that (b| blj^^,-,) G Cj(e') ®Cj^^,y but b|g, / 0. □ 

Appendix C. Proof of Forward Direction of Theorem 14.31 

Proof of (a) =^ (b) in Theorem \4.3\ Let C = Ci (Br C2 for codes Ci and C2 defined on the index sets 
Ii and I2, respectively. By definition, / = /1A/2. Set J = Ii — I2 and J = I2 — h, so that (J, J) 
forms a partition of /. In what follows, words defined on the index set Ii will be written in the form 
X = (x I J, X I PI j^); words defined on the index set /2 will be written in the form X = (xlj^^j^, x| j); 
words defined on the index set / will be written as x = (x| j, x| j); and finally, words on the index 
set Ii U I2 will be written as x = {x.\j,x.\j^^j^,x.\-j). 

We begin by proving that dim(C| j) = dim(Ci). This is accomplished by a two-step argument: 
we first show that C\j = Ci\ j, and then we show that Ci\ j = Ci. 

If a G C| J, then there exists some x G Ci, y G C2 such that (x ★ y)| j = a. However, (x * y)| j = 
x|j as J lies outside Ii fl l2- Hence, a = x[j G Ci\j. Conversely, suppose that a G Ci\j. Then, 

there exists z G c[^^ such that x = (a, z) G Ci. Since c[^^ = there exists y = (z, b) G C2. 
Now, X ★ y = (a, 0, b), and hence (a, b) G C. Thus, a G C\j, which completes the proof of the 
fact thatClj = Ci\j. 

Now, to show that Ci = Ci\j, let us consider the projection map tt : Ci — > Ci|j defined by 

7r(x) = x[j. This map is a homomorphism, with kernel isomorphic to c['^\ which is {0} by 
definition. Hence, vr is in fact an isomorphism, which proves that Ci = Ci\j. 

We have thus shown that dim (C I j) = dim(Ci). A similar argument yields the fact that dim(C|j) = 
dim(C2). Hence, 

dim(C| j) + dim(C|j) - dim(C) = dim(Ci) + dim(C2) - dim(Ci C2) = r, 
by Corollary O 

It remains to show that min{| J|, | J|} > r. Note that since dim(C| j) + dim(C| j) — dim(C) = r, 
and dim(C|j) < dim(C), we must have dim(C|j) > r. Therefore, \J\ > dim(C|j) > r. By a 
similar argument, we also have \ J\ > r. □ 

Appendix D. Proof of Proposition [STU 

The proof of Proposition 15.11 requires the following lemma, which presents a property of the 
codes Ci and C2 obtained via the r-sum decomposition procedure of Section |4] 

Lemma D.l. Let C be a code defined on the index set I, and let (J, J) be a partition of I, with 
dim(C| j)+dim(C| j)— dim(C) = r. Suppose thatCi andC2 are the codes, defined on the respective 
index sets Ii and I2, that are obtained by the procedure described in the proof of Theorem \4~3\ Then, 
for any Ji C J, and any J2 ^ J, we have 

dim(Ci|^J + dim(Ci|^^_^J - dim(Ci) = dim(C|^J + dim(C|j_^J - dim(C), (29) 
dim(C2|j^) + dim(C2|^^_jJ - dim(C2) = dim(C|jJ + dim(C|^_jJ - dim(C). (30) 
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Proof. We use notation from the proof of the (b) =^ (a) direction of Theorem 14.31 Thus, C, Ci and 
C2 are generated by the matrices G, Gi and G2 given by ( fTOl ) and ([TT]) . respectively, which we 
reproduce here for the sake of convenience. 

A O B 

AX], 



G 
Gi 
G2 



4, 
O 



[ 4, 
X 

o 



O B 

h-ki G 

For any matrix M, given a subset Z of the column indices of M, we will denote by the 
restriction of M to the columns indexed by Z. Thus, 

O O 



GI 



Gi\j=[lk, A 



G\j- G2I7 



O 

h-ki 



B 
G 



W 



Our proof of the lemma uses only elementary linear algebra. We prove (1291 ) first. Consider any 

r G I 1 

Ji C J. It is clear that G\j^ = ^-^i , and therefore, we have dim(C|jJ = rank(G|jJ = 

rank(Gi|jJ = dim(Ci|jJ. Next, note that J - Ji = (J - Ji) U J, from which we have 

dim(C|,_^J = rank([G|^_^^ G\j]) . 
Now, observe that by performing column operations on G\j, we can bring it into the form 

O B 

h-ki O 
Hence, 

rank([G|^_^^ G\j]) = rank {[G\j_j^ W]) 

= rank(4_fcj + rank ([Gil B]) 
= A;- A:i + rank([Gil_;__;^ B]). 

At this point, we have 

dim(C|^J + dim(C|,„^J = dim(Ci|^J + k-h + rank ([ Gi|^_^^ B]) , 

which upon re-arrangement yields 

dim(C|^J + dim{C\i.j,) - dim(C) = dim(Ci|^J + rank ( [ Gi|^_^^ B]) - dim(Ci). 

Thus, would be proved if we could establish that dim(Ci = rank ([ Gi\j_j_^ B]). 

Now, /i — Ji = ( J — Ji) Li Ix, and hence, 

dim(Ci|,^_^J = rank([Gi!^_,^ X]). 

Thus, we have to show that rank ( [ Gi | j„ B ])= rank ([ Gi [ X ] ) . We will prove that 
the matrices B and X have identical column-spaces. Clearly, the desired result then follows. 

Recall that for i = 1, 2, . . . , fci, the ith row of B can be uniquely expressed as a linear combina- 
tion, Yl^j=i o^jjbj, of its first r rows bi, . . . , b^. Furthermore, the ith row of X equals Y^j=i ctijdj 
for the same ai,/s, where di, . . . , are the rows of the generator matrix, Dr, of the code A^. In 
particular, the first r rows of X constitute the matrix Dr. Denote by Br the submatrix of B com- 
prised by its first r rows. 

Now, it was pointed out in Section |4] (a little after the proof of Proposition 14.11 ) that any col- 
umn vector in is a scalar multiple of some column of Dr. Therefore, any column of Br is a 
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J(e) 



J(e) 



Figure 11. Tg is a subtree of Tg. 

scalar multiple of some column of Dr- But because of the way X was constructed, this implies 
that any column of i? is a scalar multiple of some column of X. Thus, the column-space of B is 
a subspace of the column-space of X. However, we also have rank(i?) = rank(X), and so, the 
column-spaces of the two matrices are in fact identical. This proves that rank ( [ Gi\j_j^ B ] ) = 
rank([Gi|j_j^ X ]), and (Eg follows. 

To show (l30l ). consider any J2 C J. Arguments similar to the ones above establish that 

dim(C|jJ = dim(C2| jj and dim(C|^_jJ = h + rank C] . (31) 

Now, consider dim(C2|j2_j2) = '^Sink{G2\j,^_jJ- Noting that I2 — J2 = Ix ^ (J — J2), we see 
that the matrix G2\j^_j,^ has the form 

" X O B\j^ 

with /' = (/fc-fcjlj-jj-;^' for some K Q J — J2. Since the columns of B are contained in the 
column-space of X, we can perform column operations on G2\i^_j^ to bring it into the form 

X o o 



Hence, 



rank(G: 



w 



rank(H^') 



O /' C\ 



K 



rank(X) + rank([/' C\^]) 
{ki + k2-k) + x^vk{[Ik-k, 



Some trivial manipulations of (I3TI ) and (132] ) yield (1301 ). which proves the lemma. 



(32) 

□ 



Proof of Proposition \5Jl Recall that T* = {T,uj, (5^, e G E{T)), (C^,, v G V{T))), where cSe 
and are as defined in (fTTl) and ([TSl l. To show that T* is the minimal realization M^{C; T, lu), it is 
enough to show that for all e G E{T), dim(5e) equals the expression in ([S]), i.e., 

dim(5e) = dim(C|^(^)) + dim(C|j(^)) - dim{C). (33) 

Note that this is true when e = e, since dim(5e) = dim(Ar) = r, and from (fT2]) . we have 
r = dim(C| + dim(C| j^-)) - dim(C). We must therefore show that ^ holds for e G E{T) - 

{e} = E{T^) U ^(Tg). We will prove this for e G ^(Tg); the proof for e G ^(T^) is similar. 

So, consider any e G E{Te). One of the two components, Tg and Tg, of T — e is contained in 
Tg. Without loss of generality, we may assume that it is Te that is a subtree of Tg, as depicted in 
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Figured!] Hence, J(e) = uj-'^(y{Te)) C J(e). Now, by flT]), Se = S^ ', the latter being the state 
space associated with e in Al(Ci; Te, wi). Therefore, by ([5]), 

dim{SP) = dim(Ci|j(^)) + dim(Ci - dim(Ci). 

But, by Lemma ID?T1 the above expression is equal to the expression on the right-hand side of (|33] ). 
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